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ABSTRACT * , ^ 

Follow Through is a longitudinal program instituted 
by the U.S. Office of Education in which a variety of curriculum 
models have been implemented in a number of locatidns nationwide • 
Because services are provided from kindergarten through third , grade,, 
many children enter and leave the program at points not coterminous 
with the treatment itself. The purpose of this -.study w?ls- t-o assess 
the effects of attrition on the national Follow Through evaluation.' 
relationships between child characteristics such as ethnicity, sex, 
family income, and mother's education and attrition group ^membership 
were invest,igated to' determine whether the ^findings of the major 
analysis should be gualified. (Author) 
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One of dhe'^most importarft problems that is germane to the topic of 
social experimentation, educational evaluations, and especially to longitudinal 
studies is the issue of attrition, that is, the loss of units from a study. 
In a critique of educational evaluation studies including Sesame Street, 
Head Start, and Follow Through, Anderson (1973) suggested that additional 
information as to the nature of these program$ be provided to assist the 
reader in interpreting the major conclusions. Such information, which has 
largely been omitted in the past, phould include .provision of data on . 
dropouts in groups of siibjects. (Failure .to take account of differences in 
the number and kinds of dropouts in groups that are to be compajred repre- 
sents a major source of error in conclusions about t;he effects of educational 
treatments.)'*' (Anderson, 1973, p. 202). 

While the significance of attrition problems was popularized by 
Campbell and Stanley (1963) in their identification of attrition as a 
"possible rival hypothesis," later woi^k by Jurs and Glass (1970) revealed 
that mortality estimates are seldom given in reports of educational studies, 
and that virtually no textbooks include attrition as a topic of interest'. 
Although kershaw (1971) gives suggestions for countering attrition in 
interview situations, and Riecken and Boruch (1975) as well as Jurs and, 
Glass (1970) discuss various aspects of the attrition problem, it is 

clear from a review of educational studies that investigations^ of the 

L ' ' 

effects of attrition on either the internal or external validity of educa^ 

tional studies are usually lacking. 

Our general interest^ in attrition stems in part from the observation ^ 
that the infrequency of attention to attrition exacerbates problems of 
data interpretation. We are concerned that the lack of a clear understanding 
of the causes and effects of attrition will inhibit the development of be^^fe^ 
evaluation technology and will obscure the identification 6f valuable educa-*^ 
tional and social programs. ■ ' ^ 

The purpose of the present study is --to ascertain if the composition " 
of the treatment and control groups in one large-scaJL'fe longitudinai^ program, 
Follow Through, has changed differentially as a^result of attrition. 
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Follow Through is a large-scale experimental program in compensatory 
education. It was implemented through an approach known as "planned 
variation" which included thB systematic introduction of a variety of pro- 
grams into the kindergarten through third grade years of public education 
by educational specialists (sponsors) from research institutions and univer- 
sities Who each implemented their own educational model in a group of school 
districts twites) . 

The national evaluation of Follow Through, in which Abt. Associates 
Inc. (AAl) has participated since July, 1972, is designed to examine the 
effects of different approaches to education for improving the performance 
of disadvantaged children in a variety of areas. The national evaluations 
have concentrated on examining data collected froin groups of program partici- 
pants (Cohorts) in each-' year since 1969. These data include testg adminis- 
tered to children, questionnaires submitted by teachers, and interviews 
collected from parent*-. 

In a report entitled "Education ag Experimentation : /a Planned 
Variation Model. Volume Abt Associates has analyzed c'da/a collected on the 
impact o^ four years W Follow Through on participants wh6 entered -the 
program in fall, 197o\cohort II)." As a substudy, AAI Was investigated 
whether attrition of subjects from sites incluae(? in tfie national 
evaluation has'biased the evaluation. It is f ro\ th/s substudy that the 
data for the current paper has been drav^n. 

The Sample . , ' ' , 

The Cohort II "analytic- sampj/e" includea^;a total of 5 , 519- children 
(3,369 treatment and 2,152 comparison) distributed across 17 sponsors, 
wher^ each 'sponsor implemented' its- educationa/ program in between one and 
seven sites and where „each sit^ contained a /ollow Through treatment group 
(FT) and a non-Follow Through comparison gr^u^) (NFTJ , (See Figure 1.) 
the overall rate of attrition- across all ,s/onsors was. 65% ~ 6K^or FT and 
70% for.-^T. We have defined attrition U include not only those subjects 
who dropped out of the program d^e to mijillty, ^illness , etc. , but also 
' those subjects who weTe not retained in//the analytic sample due. to. missing ■ 
data, inadequate cell size, etc.; or w^ were, dropped from the program 
for administrative reasons. 
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FIGURE 1 



Sf>ONSOR SITE TREATMENT SUBJECT 
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.ta w^re^ analyzed in order to answer the following question: 
onsor , .^<^loe&>, the FT/NFT wi thin-site dif f erenc^^in rate of 
attrition di^^yL: '^^o^$ lex?^ls of pretest or income? A hierarchical 
regression anal^^d^ was'^lriuij for each sponsor to predict a dichotomous 
dependent variable (child was in the analytic sample vs, child not 
retained) . The regression model was as follows: ^ 



Where set A is composed of variables through X^, set B is composed of. 

variables X, through X , and variables are defined as follows: 
k+1 L 

Y = dichotomous dependent variable 

1 = not retained in analytic sample 
0 = retained in analytic sample 

a^ = constcuit ' * ^ 

U = unit vector 

b, ... b = regression weights for. X^ ... X 
1 L \ 1 L 

X^ = pretest 

X^ ... (number of sites - 1) orthogonal between-sites 

Helmert contrasts 

i 

X. ... X. =^ (number of sites) orthogonally .coded treatment 
1+1 T . , . . 

within-site contrasts 

.5 = FT 

-.5 = ^^FT . * \ 



X . A ... X, - interaction of X_ and X^ 

j+1 K ^ 1,2 



k+1 



P 



X '= interaction of X^ and X. . 
L 1 1+1 



X. 
1 
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An F test was performed to determine the significance of the 
incremental variance added by set\B. the last set of coefficients in the 

iteraction' of 



model (b . . . r the interaction* of pretest with treatment within-site) . 

If the overall F\ test proves non-significant, that is, if the set of inter- 

\ 2 
action variables fails to add "significantly" to R We proceed no further. 

If, however, the int^^f action terms (set B) do explain variance in the 

dependent variable, wd 'gamine the 'individual treatment within-site 

coef f icients.'for signiiic^c'i^. ' ' 

Results 



As can be seen from an examination of Tables I and II/ there are 
few sponsors in which additional variance in the -dependent variable is 
accounted for by the pretest (or income) by treatment within-site sets of 
variables. In other words, in general, .children in the analytic sample are 
representative of children who started the Follow Through program, at least 
in terms of pretest and income data. This suggest^ that the effects of 
attrition on the Cohort II FT evaluation have been minimal. 

\ 

For tho^e sponsors in which additional variance is acc^ounted for, ^ 



he individual within-site terms were tested for significanc^ (^^1 level), 
^vt should be noted that in no case is more than an additional one percent 
of the variance in the dep^dent variable explained by these teJrms. Those 
within-site regression lines of attrition rate on pfitest and att^rition 
rate on income which wer^ determined to be different for FT and NP.T are 
displayed ^n Figures 1 and 2. For instance, in Sponsor 4 Site 1, the 
attrition rate within FT is highest for lower scoring children, While it 
is relatively cons|(:ant within NFT. Examination of the other regression 
lines displayed ir^ Figures 1 and 2 shows that while ^here are only a few 
cases of dif f erent^ial attrition, there are several variations on the 
theme. For thesej sites the use of pretest an<5/or income ^s covariate^ 
becomes especially importaitt. ^ To the extent that the assumptions of the 
analysis of covariance are not met, the p.nternal validity of the evalua- 
tion within that site is called into question. 



TABLE I 

2 

INCREMENT TO R ADDED 3Y TI^E PRETEST 
BY TREATMENT WITHIN-SITE INTERACTION SET * 



Sponsor | 


R^Y-AB^ 




R^Y'A 


2 . 2 
R Y-AB-R Y-A . 


01 


.025 




.006 


.019 




02 


.013 




.013^ 


.000 




03 


.088 




' .088 


.000 




04 


,081 




.059 


.022* 




05 


.049 




.048 


.001 




06 


.065 ^ 




.064 


.001 




07 


.102 ' 




.100 


.002 




08 


.143 




.134' 


.009* 




09 


.054 




.045 


.009* 




10 ^\ 


.150 




J .149 


VOOl 




11 


" .036 




.032 


.004 ' 
.001 ' 




12 


. 141 




.140 




13 


.133 




.132 


.001 

/ f 




14 


.006 




.005 


.001 1 ' 




15 


.265 




.260 


.005* ' 




16 


.006 , 




.005^ 


.001 




17 






.073 


■ .004 
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' "V TABIiE II 

2 ' 
INCREMENT TO R ADDED BY THE INCOME 

BY TREATMENT WITHIN SITE INTERACTION SET ^ < 



Sponsor 


R^Y-AB 


R^Y-A 


R^Y-ABtR^Y-A . 


01 




.044 


.043 




.001 


02 




.014 


.010 




.004^ 


03 ■ 




.074 


^074 




.000 


04 




.087 


.086 




.001 


05 




• .041 


.038 




.003 


06 




.073 


.070 




.003 


07 




.101 


. 100 




.001 


08 




.129 


.125 




.004 


09 




.037 


.031 




.006 


10 




.146 


.137 




.009* 


11 




.0.43 


.039 




.004 


12 




.128 


.127 




.001 


13 




.129 


.123 ' 




. <?06* • 


14 




.004 


.002 




.002 * 


15 




.254 ' 


.250' 




- 004 


16 




.014 


.006 




.008* 

< 


17 




.080 


%078 




.002 



1 , ' 
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Figure ^ ' 

Regression Lines of Attrition Rate on Pretest'for FT and NFT Groups 
in the Sites where the Pretest by Treatment Within-Site Regression 
Coefficient is Statistically Significant at the .01 Level 
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Figure 2 

Regression Lines of Attrition Riate on Income for FT and NFT Grpups 
in Sites where the Pretest by Treatment Within^Site Regression 
Coefficient is Statistically Significant at the .01 Level 

SPONSOR 10, SITE 1 
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